Video presentation device and method

ABSTRACT

A computing device may include a video input interface and video output interface to allow insertion into a video system. The device may be configured to perform passthrough operation on the incoming video, allowing the incoming signal to appear at the output interface. When overlay video operations are desired, the device may resize the incoming video, generate one or more additional overlays, and generate a modified version of the incoming video signal, where the modified version contains the one or more additional overlays along with (if desired) the original video. The device may have data network connections, and can thereby serve as an easy way, for example, to upgrade an existing video system with network and/or interactive features.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional ApplicationNo. 61/637,487, filed Apr. 24, 2012, and entitled “Video AdapterDevice,” the entire contents of which are incorporated herein byreference as an example embodiment.

BACKGROUND

The availability of new information and entertainment services andfeatures continues to grow at a rapid pace, and the latest electronicdevices are able to offer a myriad of new options to customers.Electronic hardware, however, is not as easy to update, and there areplenty of users who have resisted getting new stereos, DVD (DigitalVersatile Disc) players, gaming consoles and televisions. There remainsan ever-present need to support such legacy devices, even as newservices and features are introduced.

SUMMARY

Some of the features described herein relate generally to a videoadapter device, and associated methods of use, that can serve as apassthrough device for an existing video display signal, and which canoverlay additional information and graphics obtained via a network dataconnection. In one embodiment, the device may include an input videointerface conforming to a video standard; an output video interfaceconforming to the video standard; a network connection; a processor; anda memory, storing instructions that, when executed by the processor,cause the device to perform the following: conduct passthrough videohandling of an input video signal; generate one or more video overlaygraphics using data obtained over the network connection; combine theone or more video overlay graphics with the input video signal togenerate a modified version of the input video signal; and provide themodified version of the input video signal to the output videointerface.

In some aspects, the network connection may be an Internet connection,and the overlay can include data obtained from the Internet connection.In some embodiments, the original video received at the input videointerface may be resized in the modified version of the input videosignal, and the resized video may be displayed adjacent to the overlayin a separate graphical panel. In some embodiments, the overlay graphicscan include data obtained from a security system at the same premises(e.g., a home security system), as well as video obtained from a localcamera communicating with the adapter device.

In some embodiments, the camera may be used for facial recognition anduser profile login, and can be used to engage in video chats withfriends of a user. The video chats may include sharing of content, suchas picture images stored on a user's thumbdrive, which may be attachedto the adapter device.

In some embodiments, video passthrough handling may be conducted toallow the same input video signal to appear at the output videointerface, but when an overlay is needed, such as when an incoming videocall invitation is received, the adapter can resize the original inputimage and generate an overlay using incoming images associated with thevideo call.

In some embodiments, a user may employ a secondary device, such as asmartphone or tablet computer, to view a copied version of the modifiedversion of the input signal, and use the secondary device to manipulateand rearrange the various graphics and panels appearing in the modifiedversion of the input signal. Such manipulation may also be performedusing in-air gestures, such as hand motions, detected by the camera. Forexample, the space in front of a user may be divided into differentregions corresponding to different graphical panels, and the user mayposition hands in the regions and make predefined gestures to enter acommand for the region's panel.

The adapter device (which may be implemented, for example, by hardwareand/or software) may be installed to upgrade the interactivecapabilities of an existing video output device. The installation mayinvolve identifying the original output device's output line,disconnecting it from the display device and connecting it to acorresponding input on the adapter device instead, and connecting asecond video line between an output interface on the adapter and thevideo input interface on the display device to which the original outputline was connected. The adapter can then switch between a passthroughmode in which an original incoming video signal is simply passed throughto its output, and an overlay mode in which one or more graphicaloverlay objects are added to the video signal before sending it to theoutput. The camera may be installed as well, and may be used asdescribed above to provide the various features described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example information distribution network on whichmany of the various features described herein may be implemented.

FIG. 2 illustrates an example physical appearance of a video adapter.

FIG. 3 illustrates an example configuration using video adapter.

FIG. 4 illustrates an example hardware schematic for the video adapterof FIG. 2, showing more detail.

FIGS. 5 a-10 illustrate example display screens.

FIGS. 11 a&b illustrate an example method of installing and using thevideo adapter.

DETAILED DESCRIPTION

In the following description of various illustrative embodiments,reference is made to the accompanying drawings, which form a parthereof, and in which is shown, by way of illustration, variousembodiments in which aspects of the disclosure may be practiced. It isto be understood that other embodiments may be utilized, and structuraland functional modifications may be made, without departing from thescope of the present disclosure.

FIG. 1 illustrates an example information distribution network 100 onwhich many of the various features described herein may be implemented.Network 100 may be any type of information distribution network, such assatellite, telephone, cellular, wireless, etc. One example may be awireless network, an optical fiber network, a coaxial cable networkand/or a hybrid fiber/coax (HFC) distribution network. Such networks 100use a series of interconnected communication links 101 (e.g., coaxialcables, optical fibers, wireless, etc.) to connect multiple premises 102(e.g., businesses, homes, consumer dwellings, etc.) to a local office103 (e.g., a headend, a processing facility, etc.). The local office 103may transmit downstream information signals onto the links 101, and eachpremises 102 may have a receiver used to receive and process thosesignals.

There may be one link 101 originating from the local office 103, and itmay be split a number of times to distribute the signal to variouspremises 102 in the vicinity (which may be many miles) of the localoffice 103. The links 101 may include components not illustrated, suchas splitters, filters, amplifiers, etc. to help convey the signalclearly, but in general each split introduces a bit of signaldegradation. Portions of the links 101 may also be implemented withfiber-optic cable, while other portions may be implemented with coaxialcable, other lines, or wireless communication paths.

The local office 103 may include a termination system (TS) 104, such asa cable modem termination system (CMTS) in an example of an HFC-typenetwork, which may be a computing device configured to managecommunications between devices on the network of links 101 and backenddevices such as servers 105-107 (to be discussed further below). In theexample of an HFC-type network, the MTS may be as specified in astandard, such as the Data Over Cable Service Interface Specification(DOCSIS) standard, published by Cable Television Laboratories, Inc.(a.k.a. CableLabs), or it may be a similar or modified device instead.The MTS may be configured to place data on one or more downstreamfrequencies to be received by modems at the various premises 102, and toreceive upstream communications from those modems on one or moreupstream frequencies. The local office 103 may also include one or morenetwork interfaces 108, which can permit the local office 103 tocommunicate with various other external networks 109. These networks 109may include, for example, Internet Protocol (IP) networks Internetdevices, telephone networks, cellular telephone networks, fiber opticnetworks, local wireless networks (e.g., WiMAX), satellite networks, andany other desired network, and the interface 108 may include thecorresponding circuitry needed to communicate on the network 109, and toother devices on the network such as a cellular telephone network andits corresponding cell phones.

As noted above, the local office 103 may include a variety of servers105-107 that may be configured to perform various functions. Forexample, the central office 103 may include a push notification server105. The push notification server 105 may generate push notifications todeliver data and/or commands to the various premises 102 in the network(or more specifically, to the devices in the premises 102 that areconfigured to detect such notifications). The central office 103 mayalso include a content server 106. The content server 106 may be one ormore computing devices that are configured to provide content to usersin the homes. This content may be, for example, video on demand movies,television programs, songs, services, information, text listings, etc.In some embodiments, the content server 106 may include software tovalidate (or initiate the validation of) user identities andentitlements, locate and retrieve (or initiate the locating andretrieval of) requested content, encrypt the content, and initiatedelivery (e.g., streaming, transmitting via a series of contentfragments) of the content to the requesting user and/or device.

The local office 103 may also include one or more application servers107. An application server 107 may be a computing device configured tooffer any desired service, and may run various languages and operatingsystems (e.g., servlets and JSP pages running on Tomcat/MySQL, OSX, BSD,Ubuntu, Redhat, HTML5, JavaScript, AJAX and COMET). For example, anapplication server may be responsible for collecting television programlistings information and generating a data download for electronicprogram guide listings. Another application server may be responsiblefor monitoring user viewing habits and collecting that information foruse in selecting advertisements. Another application server may beresponsible for formatting and inserting advertisements in a videostream and/or content item being transmitted to the premises 102.

An example premises 102 a may include an interface 110 (such as a modem,or another receiver and/or transmitter device suitable for a particularnetwork), which may include transmitters and receivers used tocommunicate on the links 101 and with the local office 103. Theinterface 110 may be, for example, a coaxial cable modem (for coaxialcable lines 101), a fiber interface node (for fiber optic lines 101), orany other desired modem device. The interface 110 may be connected to,or be a part of, a gateway interface device 111. The gateway interfacedevice 111 may be a computing device that communicates with theinterface 110 to allow one or more other devices in the home tocommunicate with the local office 103 and other devices beyond the localoffice. The gateway 111 may be a set-top box (STB), digital videorecorder (DVR), computer server, or any other desired computing device.The gateway 111 may also include (not shown) local network interfaces toprovide communication signals to other devices in the home (e.g., userdevices), such as televisions 112, additional STBs 113, personalcomputers 114, laptop computers 115, wireless devices 116 (wirelesslaptops and netbooks, mobile phones, mobile televisions, personaldigital assistants (PDA), etc.), and any other desired devices. Examplesof the local network interfaces may include Multimedia Over CoaxAlliance (MoCA) interfaces, Ethernet interfaces, universal serial bus(USB) interfaces, wireless interfaces (e.g., IEEE 802.11), Bluetoothinterfaces, and others.

FIG. 2 illustrates an example physical appearance of a video adapter201. The adapter 201 may include a variety of indicators on the frontpanel 201 a, such as light emitting diodes (LEDs) to indicate networkconnectivity, chat message availability, missed incoming messages,power, volume, or any other desired visual indication of operation. Therear panel 201 b may include a variety of ports and interfaces tocommunicate with external systems and devices. For example, the rearpanel 201 b may include an Ethernet interface 202 configured tocommunicate on an Ethernet network device. For example, the Ethernetinterface may be coupled to a corresponding port on a gateway device111.

The adapter 201 may include one or more pairs of signal passthroughconnections 203. For example, the connections 203 may include a pair ofinput and output HDMI (High Definition Multimedia Interface) ports. Inoperation, the signal arriving on the input port can include a videosignal (e.g., the video portion of an HDMI signal), and the componentswithin the adapter can process this signal and add one or more overlayvideo (and/or audio) signals, and output an output signal that is in thesame format as the input (e.g., another HDMI signal), but containing theoverlaid signals. The overlaid video and/or audio signals can be used toenhance a user's viewing experience, as described in greater detailbelow. Although HDMI ports are used as examples, any desired type ofsignal line can be used. For example, video composite, component,digital video interface (DVI), coaxial cable, wireless, Ethernet, USB,etc. may be used.

The adapter 201 may include one or more ports 204 for local devices,such as Universal Serial Bus (USB) ports that can be used to communicatewith any number of desired local (or remote) devices. The adapter 201may also include one or more audio output (or input) ports 205. Forexample, a digital audio output port can provide an optical outputsignal containing digital audio.

FIG. 3 illustrates an example configuration using video adapter 201. Inthe example configuration, a user may have a video device 301, such as atelevision or video monitor, that is used to view video content. Thevideo content may come from a video source, such as a digital videorecorder (DVR) 302. The video source 302 may use a video connection 303,such as an HDMI cable, to provide a video output signal (audio may alsobe included in any reference to video signals herein). Without theadapter, this video connection may be plugged directly into an HDMIinput port on the back of the television 301. However, upon installationof the video adapter, the video connection 303 may be inserted into the“input” port 304 on the adapter, while another video connection 304, inthe same format as the input (e.g., in HDMI format) can supply themodified video signal to the television 301 for display. In thispass-through manner, the adapter 201 can be installed in-line in thevideo line that the user is already using to receive video signals onthe television 301, and installation can be easily done.

The adapter 201, however, may also be connected to other devices. Forexample, the adapter may be connected (e.g., via a USB line 204) to acamera 306. The camera 306 may capture moving and/or still video images,and can further process them to perform gesture recognition, facialrecognition, bar code or symbol recognition, or any other desired typeof camera input.

Another example may be a control line used to control other devices. Forexample, the adapter 201 may be connected to an infrared emitting diode(not shown), which can then be used to transmit infrared commands tosimulate those sent from a typical infrared remote control. In this way,the adapter can be configured to control the operation of devices suchas the DVR 302 and/or the television 301. While infrared is the exampledescribed above, other forms of remote control (e.g., radio frequency,wired) may also be used.

As another example, the adapter may be connected to a network, such as ahome's wireless network, an Ethernet network, a MoCA (Multimedia overCoax Alliance) network, etc., and may send and receive data using such anetwork to external networks, such as the Internet. The data receivedfrom these networks may be used to manipulate the incoming video signal,resulting in an output video signal that can include the manipulatedincoming signal along with added details, such as a video overlay.

The example in FIG. 3 is just an example, and the various components maybe combined/subdivided as desired. For example, the camera 306 may beincluded in a common chassis as the adapter 201.

FIG. 4 illustrates an example hardware schematic for the adapter 201,showing more detail. At its heart, the adapter 201 may be or otherwiseinclude a computing device, and may contain one or more processors 401that can execute instructions stored in a computer-readable memory orstorage device, such as a random access memory (RAM) 402 and/or flash403, to perform any of the functions described herein.

As noted above, the adapter 201 may include an HDMI (or other videosignal) input 404 and output 405, allowing passthrough of the HDMIsignal from input 404 to output 405, but with added overlay content (ifdesired), as will be described further below. The passthrough circuitrymay include an HDMI and H.264 SVC decoder and encoder.

The adapter 201 may include a variety of network interfaces, to send andreceive data that can affect the overlay content and use of the input404 signal. For example, the adapter 201 may include an Ethernetinterface 406, a Wi-Fi interface 407 (e.g., IEEE 802.11), a Zigbee/RF4CEinterface 408, and any other desired type of network interface (e.g., aMultimedia over Coaxial Alliance—MoCA—interface). In some embodiments,these interfaces (e.g., the Wi-Fi interface 407) may serve as wirelessrange extenders, extending the range of coverage for a particularnetwork type.

The adapter 201 may include removable computer-readable storage, such asvia a secure digital (SD) interface 409, which can also store executableinstructions for the processor 401.

The adapter 201 may include an infrared input and/or output interface410, which can allow the adapter 201 to receive remote control commands,as well as issue infrared remote control commands using an infraredlight emitting diode. Similarly, the interface 410 can include radiofrequency (RF) circuitry to communicate with RF-based remote controls.

The adapter 201 may include one or more additional local deviceinterfaces, such as USB ports 411, which can be used to communicate withany desired type of local device, such as camera 306. The camera 306 mayinclude its own processing and storage capability to recognize facesand/or gestures (e.g. for use in facial detection and user profilesign-in), process images to improve quality, perform voice recognitionusing built-in microphones, etc. Alternatively, such functionality maybe provided by the processor 401 in the video adapter 201, or evenremotely at, for example, an app server 107. In some embodiments, thecamera 306 may include a privacy shutter, which can physically cover upone or more lenses when the camera is not being used, and a front panelindicator (e.g., an LED) can indicate that the shutter is closed, togive users some comfort in knowing that they are not being secretlyvideo recorded.

For processing audio, the adapter 201 may include an audio line out 412,which can be a digital optical audio output. The adapter 201 may alsoinclude a speaker and/or microphone 413, allowing the device to locallytransmit and/or receive audible sounds in the room. In some embodiments,the processor 401 may be configured to perform text-to-speech conversionto audibly announce the contents of incoming text messages and pop-upalerts. Similarly, the processor 401 may be configured to respond tovoice commands.

FIGS. 5 a&b illustrate example display screens. FIG. 5 a illustrates anexample video screen image 500 that may be supplied by a video source,such as DVR 302, on a video line such as HDMI line 303. The image 500may be passed in to the adapter 201, and upon emergence the output line304 may carry a modified version of the original source video, havingscreen image as shown in FIG. 5 b, which includes the original screen500 with an overlay image 501. The overlay image 501 can be of anydesired type. For example, the image 501 can contain Internet content,such as an Internet page or a streaming Internet video. The overlayimage 501 can contain video signals captured, for example, by anattached camera 306, or delivered locally to the adapter 201 viaEthernet, Wi-Fi, etc. from a user's device such as a smartphone, tabletcomputer, desktop computer, etc. In some embodiments, the overlay image501 can swap places with the primary image 500, and occupy the largerportion of the screen while the original screen image appears in thecorner. This may be useful, for example, to allow a user to surf anInternet page on the larger area, while still keeping an eye on thehappenings of a television program in the overlay area.

FIG. 6 a illustrates another example, in which the user has a primaryviewing area of the screen 600, and three separate sub-panels 601 openon top of the primary one. These areas can be populated with anycombination of the original input signal from the video source (e.g.,from the DVR) and signals received from other sources, such as theInternet, local tablet computers, etc. So, for example, the FIG. 6 aimage may have an Internet browser in the main screen 600, the videosignal from the DVR in one of the sub-panels 601, and two otherstreaming Internet videos in the other sub-panels. As another example, aringing of the doorbell can cause activation of a front door securitycamera (not shown), which can then deliver a video stream to the adapter201, which can then overlay a sub-panel 601 so the user can see who isat the door without even getting up from the couch.

The user may receive a request 602 to conduct a video chat or videocall. Users of the adapter may initiate and send invitations to viewcontent and engage in chats by, for example, accessing a socialconnection application in memory (e.g., in RAM), and request that theapplication send a message to an identified contact of the user. Notethat the social connection application can interact with any desiredsource of contact, such as sending an Internet request to a server thatruns a social networking website, identifying the user and requestingthe user's friend list. In this way, the adapter 201 can interact with avariety of different address books for the user. In some embodiments,the adapter 201 can query a presence-aware application to identifyfriends who are watching a show (e.g., if that presence-awareapplication included a login feature, facial recognition, or othermechanism for determining who is actually in the room), and then use theaddress book from a different social network application to request IPaddresses for the friends who are watching the show, in order to delivera chat invite to those friends. As an example, the adapter 201 may use auser's SKYPE™ buddy list to identify people who are online, and then usethat list to request addressing information from a PLAXO™ database todeliver messages and invitations.

The user may see the chat request 602, and may accept it by pressing abutton on a remote, entering a gesture involving hands/arms/head,speaking “yes” out loud, or any other desired input mechanism.

Upon initiating the chat, the display may be rearranged and the panelsmay be rescaled. FIG. 6 b illustrates an example in which adapter 201 isused to carry an audio and/or textual chat, and the panels may berearranged. One or more of the panels (e.g., a sub-panel 601) may bechanged to display a video image from one of the participants to thechat. A separate chat window 603 may appear, and may be used to enterand receive chat messages. The user may use any desired device, such asa laptop's keyboard, a keyboard on an RF remote control of the adapter201, or a tablet connected via Wi-Fi to communicate with the adapter 201and enter text. Although a text chat is discussed as an example, thechat may be a video call, in which case the microphone and speaker 413may be used to conduct the discussion, instead of (or in addition to)the text entry panels shown.

The user may further rearrange the panels, such as that illustrated inFIG. 7. The repositioning can be controlled using camera-detectedgestures. For example, the user may hold a hand in a predeterminedposition to sift or page through the various onscreen elements (whichmay be thought of as “layers” in the image), and may highlight one tobring it forward, move it to a different location, identify it as atarget for future keyboard inputs, close it, etc.

The panels may be used to conduct a social media viewing experience. Forexample, a user can connect an SD memory card, tablet computer,smartphone, digital hand-held camera, or other device to the adapter201, and begin a joint picture-viewing session with a chat friend. Asillustrate in FIG. 8, the user may be given a primary panel 800 in whicha picture is displayed, and the image for this panel 800 may be takenfrom the connected picture device. Overlaid on that image may be animage 801 from the user's chat camera 306, and an image 802 from a chatfriend's camera, as well as navigation controls 803 for paging throughthe displayed pictures.

The user's adapter 201 may transmit the photo images as a stream to thefriend's corresponding adapter 201, and may transmit video images fromthe user's camera 306 in a different stream. The friend's correspondingadapter 201 may receive, assemble and rearrange the various panels asthe friend desires.

In some embodiments, the participants may be given additional commandoptions. For example, the chatting friend may be given an option toenter a command (e.g., by speaking a voice command, or selecting anonscreen icon) to copy an underlying photo to that friend's own storage,such as the friend's own SD card, a cloud storage, or another computingdevice communicating with the friend's adapter 201. In this way, parentsof a newborn can share photographs of the child with grandparents, andcan engage in a joint viewing experience with voice chat to discuss theimages, and the grandparents may have the option to automatically tagpictures for copying and/or local storage to their own computingdevices. In some embodiments, the various photographs may be uploaded toa server, such as app server 107, and the app server 107 may process thephotographs to generate a video file (e.g., an MPEG video file) that canthen be made available to users for download and/or streaming on demand.

Similarly, since additional sub-panels may be displayed, the chatparticipants may also add one or more additional sub-panels to theirdisplay, without alerting the other chat participant. So, for example,during the baby picture viewing with the grandparents, another user canhave an overlay of local DVR content appear onscreen, and can watch atelevision program while also chatting with the grandparents. Of course,the grandparents can also do the same.

A pop-up alert, such as the chat invite 602, can be used to alert theuser of a variety of things, in addition to an incoming chat or callrequest. For example, a similar message may be displayed when thedoorbell rings (as discussed above), when a breaking news story occurs,or when an incoming telephone call is received. Any desired news feed orIP stream can serve as a trigger to causing a pop-up alert to appear.

The joint viewing session of FIG. 8 can be used for more than simplyviewing pictures. For example, multiple friends may join a common chatto watch a particular movie, and their various adapters 201 maycoordinate synchronization of the different streams so that latency anddata traffic delay can be accounted for, so that the users all see thesame content at the same time in a coordinated viewing experience. Eachparticipant's video device, such as a DVR, may initially receive astream of the underlying movie itself, and the various adapters 201 maybe responsible for gathering synchronization information, and forsending pause/playback commands to their respective DVRs to cause thesynchronized playback.

In such a joint viewing experience, users can press a button to tagcertain portions of the program and provide their own written commentsthereon, or to tag them for inclusion in a version of the program thatcontains only the portions tagged by friends during the viewing.

In some embodiments, a user may enter a command to request that the chatsession be recorded on the adapter 201 or another device (e.g., aseparate tablet computer or desktop computer networked with the adapter201). In response to such a command, the adapter 201 may initiate astream of its output video to the target recording device. For example,a user can have a virtual visit with a doctor as a chat, and can havethe examination session recorded for later inclusion in the doctor'sfiles. The chat can also include additional sub-panel overlays 601containing the patient's vital statistics, detected measurements, or anyother data that is relevant to the visit. That other data may beretrieved, for example, via an Internet connection between the adapter201 and an Internet server.

In some embodiments, when a user is engaged in a voice chat, the adapter201 may be configured to detect the audio of the program being watched,and filter out that audio from audio detected by the microphone 413 andsent to the other users in the chat. In this manner, the other user neednot receive or hear the audio from the first user's television.

FIG. 9 illustrates an example default screen that may be displayed forthe user. In it, the adapter 201 may take the original video image 600from the DVR, and resize/overlay it to generate additionaluser-selectable elements for other applications, such as an e-mailapplication, weather application, calendar application, home securityapplication, program description application, etc. The layout shown inFIG. 9 may be a default layout, and it may be customized by the user.

In some embodiments, the user may use a secondary device, such as atablet computer, to control the overlaying of images by the adapters201. As illustrated in FIG. 10, the user's television may display afirst image 1001, having an arrangement of panels similar to that shownin FIG. 9. This image may be generated by the adapter 201, which resizesthe original input image and add the other panels for the otherapplications, and the adapter 201 may also transmit an alternate versionof this image to the user's tablet 1002. The alternate version may beidentical to the original, or it may be smaller in dimension and/orresolution, omitting some graphical detail found in the original tosimplify transmission and reduce latency. For example, the alternatescreen on tablet 1002 may simply have outlines for the different panels,without labeling or graphics found in the original image on thetelevision 1001. The user may use the input capabilities of the tablet1002 (e.g., a touch screen) to rearrange and/or resize the panelsappearing on the tablet's 1002 display, and the corresponding change inposition and/or size can be replicated in the image shown on thetelevision 1001.

As noted above, in some embodiments, user gestures and/or movements maybe detected by the camera 306, and used as control inputs to the adapter201. The gestures may allow a variety of functionality. For example, the3-D space in front of a user may be detected by the camera. For example,the camera may determine that the user is 10.0 feet away from thetelevision. That space may be allocated to different panels appearing onthe display, and gestures inputted in the allocated space may bedirected to the corresponding panel or its supporting application. So,for example, if the FIG. 6 a display is shown on the user's television,the adapter 201 may determine that gestures made with 0.0-2.0 feet infront of the television will correspond to the topmost sub-panel 601;gestures made with 2.0-3.0 feet will correspond to the middle sub-panel601; and gestures made within 3.0-4.0 feet will correspond to the lowestsub-panel 601. A predefined gesture may have a meaning (e.g., a certainhand motion to signify “yes”), and when the gesture is made by the userwithin the predefined distance range from the television, the adapters201 may send the detected gesture or command to the correspondingapplication. So, for example, a “yes” gesture made 3.2 feet from thetelevision may be sent to the lowest sub-panel 601.

While the above example divides the space according to distance from thetelevision, other divisions are possible. For example, a 2-D planarspace in front of the user (and parallel to the television) may bemapped to the different regions of the screen, and gestures made in themapped region in front of the user can be sent to the application whoseimage appears in the corresponding region on the television.

FIGS. 11 a&b illustrate an example method of installing and using thevideo adapter device described herein. In step 1101, an existing videoline may be identified by the user, homeowner, service technician, orother person handling the installation. As noted above, the existingvideo line may be an HDMI data cable connected between a video source(e.g., a DVR) and a video display (e.g., the user's High DefinitionTelevision (HDTV) set). In step 1102, the user may insert the videoadapter in the video line between the video source and the display. Thismay be accomplished, for example, by disconnecting the existing videoline from the display's input terminal (not shown), and connecting it tothe adapter's video input interface 404, and adding a new video linebetween the adapter's video output interface 405 and the display's inputterminal. As an alternative, the new line may be used between the videosource and the video adapter, and the existing video line may be usedbetween the video adapter and the display.

In step 1103, the video adapter may be connected to a local datanetwork, such as a wi-fi network, an Ethernet network, a MoCA network,or any other data network available at the premises 102 a. Thisconnection may be performed by the adapter automatically upon startup(e.g., it may automatically join an Ethernet network upon connection andpowerup), or it may be performed manually by the user (e.g., the usermay activate a wi-fi radio and enter login information to have theadapter join the network). The connection may be local, and may connectto a gateway 111 or interface 110 to communicate with a further externalnetwork 101. In alternative embodiments, the adapter may include its owninterface 110 or connection to an external network, bypassing thegateway 111.

In step 1101, the adapter may connect to local camera 306. This mayoccur automatically upon the camera 306 being plugged in, for example,to the adapter's USB port 411.

In step 1105, the adapter may determine whether any overlay is needed byany software application running on the video adapter (or remotely andcommunicatively connected to the video adapter). The overlays may begenerated by a variety of applications, such as the voice chat, homesecurity and weather applications described above and depicted in thefigures. If no overlays are needed, then in step 1106, the video adaptermay simply conduct passthrough handling of the incoming video signalreceived at the input interface 404 from the video source. Thepassthrough handling may be as simple as allowing the original incomingvideo signals to pass straight through the adapter to arrive at theadapter's output interface 405, as if the adapter were a simple videoline.

If an overlay is needed in step 1105, then in step 1107, the adapter mayperform steps to gather information necessary for the display. Thisgathering of information can vary depending on the type of overlayneeded. In some embodiments, a software application on the video adapter(e.g., a video chat application) may gather information for the overlayby sending and receiving Internet data over the data network. Forexample, the video chat application may open an Internet streamingsession with an external video chat server (e.g., server 107), which maygather video from other participant(s) in the chat and make themavailable for streaming. The streaming content may be in a video format(e.g., MPEG2), which the video adapter can receive, adjust as needed(e.g., resizing) and decode for display. Alternatively, the content fromthe external server may simply be data in another format (e.g., adatabase, data packets, etc.), which the adapter (using thecorresponding software application) may use to generate the overlay. Forexample, a weather application might simply receive a few packets ofdata from the external server, indicating the current weatherconditions, and the weather application on the adapter may beresponsible for generating the video overlay screen depicting theweather conditions, graphical images of the sun, etc.

In some embodiments, the overlay can be generated by the adapter withoutneed for any data from an external server. For example, a homethermostat control application may reside on the video adapter's memory,and its various displays may be generated through use of the data storedin the memory (and the thermostat sensors in the premises).

In step 1108, the necessary overlay(s) is generated by the videoadapter. This generation may include resizing and/or rearranging theoriginal input video signal from the video source and/or the overlay(s),and preparing an output video signal for the output interface 405. Thisoutput video signal may be in the same format (e.g., HDMI) as the inputvideo signal.

The discussion above uses visual overlays as an example, but other typesof overlays may be generated as well. For example, an application maygenerate an audio overlay to add an audio signal to the audio of theoriginal input signal. The video line may carry audio signals as well(e.g., an HDMI cable carries audio and video), or audio lines may behandled akin to the video lines discussed above.

In step 1109, the video adapter may determine whether second-screenoperation is active. Second-screen operation may involve the useremploying another display device, such as a tablet computer orsmartphone, in conjunction with the primary screen shown on the display.An example of this is discussed above with respect to FIG. 10, in whichthe user employs a secondary device 1002 to control and interact withthe overlays on the display. Second-screen operation may be requested ina variety of ways. For example, the user may press a button on atelevision or DVR remote control, or on the adapter itself.

If second-screen operation is active, then in step 1110, the adapter mayregister the one or more second screen devices that will be used. Theregistration may include informing the video adapter of the address anddevice type of the second screen device (e.g., its IP or MAC address,and an identification of the device's capabilities, model number, etc.).This registration may be omitted, of course, if the second screen deviceis already registered for an active second screen operation.

In step 1111, the video adapter may then transmit the correspondingsecond screen display data to the second screen device. This may bedone, for example, using local wi-fi communication if both are connectedto the same wireless network. If they are on different networks (e.g.,the video adapter is wired through Ethernet and the second screen tabletcomputer is connected to a wireless network), then any desired networkcommunication method may be used to transmit the data. For example, thevideo adapter may transmit the data to an intermediate router, to whichthe second screen device is also communicatively connected.

In step 1112, the video adapter may determine whether a command has beenreceived from the second screen device. For example, the user of thesecond screen device may view the overlays on the second screen, and mayenter a command (e.g., by tapping or dragging on a tablet's touchsurface) to open, close, resize and/or rearrange the various overlays onthe main display by interacting with the reduced-resolution versionsdisplayed on the second screen. If an input command has been received,then in step 1113, the second screen device may transmit the command tothe video adapter and addressed to the corresponding application, andthe video adapter may then relay the command to the application forfurther processing. As noted above, this application may be local to thevideo adapter, or it may reside on an external server.

After sending any detected command, or if no second screen operation isactive, the process may proceed to step 1114. In step 1114, the videoadapter may determine whether gesture-based interaction is active. Thismay occur, for example, if a camera 306 has been installed, and has beenconfigured to recognize gesture-based commands. If gesture-basedinteraction is active, then in step 1115, the video adapter may use thecamera to map the available space in front of the camera and/ortelevision. As discussed above, different areas of the space (e.g., 0-2feet in front of the display, the upper-left corner of the 2-D plane infront of the user, etc.) may be designed by the video adapter ascorresponding to different regions of the display, or different layersof overlays.

In step 1116, the adapter (or the camera 306) may check to determinewhether a gesture has been detected. This may be accomplished using anydesired gesture detection software and/or hardware. The gestures may beany type of gesture that can be detected. For example, a hand gesturesuch as a thumbs up or thumbs down, or an arm waving gesture, may beused to signal different commands.

If a gesture is detected, then in step 1117, the corresponding commandfor the gesture may be supplied to the video adapter, which may thendetermine which command and application correspond with the gesture. Thevideo adapter can then forward the command to the appropriateapplication. The involvement of the camera and video adapter can vary.For example, in some embodiments the camera and video adapter may simplycapture a still or moving image, and deliver the image to a remoteserver for processing to detect gestures. Alternatively, the camera 306itself may include processing (similar to the processor and memory ofthe adapter) to identify gestures and match the gesture to thecorresponding application based on, for example, the space mappingdescribed above. In that example, the command sent in step 117 may bejust a command, similar to that sent in step 1113, or to a command sentfrom a remote control.

If no gesture is detected, or if gesture detection is inactive, theprocess may return to step 1105 to repeat the process.

The various features described above may be carried out by the videoadapter 201, whose processor 401 may execute software applicationinstructions stored in a memory (e.g., RAM 402, external memory, etc.)to provide, perform, and/or control the features. These variousapplications may be made available for purchase and/or download via acomputing device, such as app server 107, which may download additionalapplications to the adapter 201 upon user request. The additionalapplications may provide any number of additional sub-panels 601 andinteractive features. In some embodiments, the software may residepartially (or totally) at the app server 107, such that the videoadapter 201 may need only be a remote client terminal or “dumb box” thatconveys user inputs and renders displays supplied to it from an externalsource, such as the app server 107.

In some embodiments, the video adapter 201 may include an application toassist in coordinating the automatic synchronization of data acrossvarious devices. For example, when portable devices connect with awireless router within the home, the devices may register with anapplication on the video adapter 201, and may exchange informationidentifying new data that should be updated. The new data can be anydata (e.g., new photographs from a smartphone's camera) that was newlygenerated since a previous synchronization, and may be on the connectingportable device or any other device that has previously connected to thewireless network. The devices sharing data need not all be portable. Forexample, devices like a user's DVR, desktop computer, home securitysystem, etc. may also participate and share their data.

In some embodiments, various security cameras may be added to a user'spremises security system. Each camera may communicate with the adapter201 to provide the adapter 201 with video images and other data capturedby the camera. These various images (and audio, if desired) may beincluded in the display sent to the user's display device. For example,the FIG. 9 screen's Home Security Update may include one or more videoimages from a camera in the home. So in this embodiment, a user watchingtelevision may enter a command (e.g., a predefined gesture or remotecontrol button) to cause a particular camera's video to appear atop orbeside the television video signal. Such a video can automaticallyappear in response to a security incident. For example, a motiondetector outside can automatically send a video image from an outsidecamera in response to detecting motion, and have that video overlaid onthe video being watching by the user.

In some embodiments, an application may be used for thermostat control.The user may view an overlay containing home thermostat information,such as current temperature and settings. The user can interact withthis app (using, for example, a remote control, connected smartphone, orin-air body gestures detected by camera) to view and change homethermostat settings.

One or more aspects of the disclosure may be embodied in acomputer-usable data and/or computer-executable instructions, such as inone or more program modules, executed by one or more computers or otherdevices. Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types when executed by a processor ina computer or other data processing device. The computer executableinstructions may be stored on one or more computer readable media suchas a hard disk, optical disk, removable storage media, solid statememory, RAM, etc. As will be appreciated by one of skill in the art, thefunctionality of the program modules may be combined or distributed asdesired in various embodiments. In addition, the functionality may beembodied in whole or in part in firmware or hardware equivalents such asintegrated circuits, field programmable gate arrays (FPGA), and thelike. Particular data structures may be used to more effectivelyimplement one or more aspects of the invention, and such data structuresare contemplated within the scope of computer executable instructionsand computer-usable data described herein.

The descriptions above are merely example embodiments of variousconcepts. They may be rearranged/divided/combined as desired, and one ormore components or steps may be added or removed without departing fromthe spirit of the present disclosure. The scope of this patent shouldonly be determined by the claims that follow.

What is claimed is:
 1. A method comprising: sending, by a computingdevice and to a display device, a signal comprising at least a first anda second graphical panel associated with at least a first and secondapplication, respectively; defining a three-dimensional space in frontof a user and the display device, wherein the three-dimensional space isdivided into a plurality of sub-spaces comprising at least a firstsubspace and a second subspace, wherein the first subspace is closer tothe display device than the second subspace; associating the first andsecond applications to the first subspace and the second subspace,respectively; detecting a gesture made by the user in the firstsubspace, wherein the gesture is associated with a pre-defined command;and sending the pre-defined command associated with the gesture to thefirst application corresponding to the first subspace in which thegesture was detected.
 2. The method of claim 1, further comprising:receiving video images from a camera; and generating one or more overlayvideo images within the first graphical panel.
 3. The method of claim 1,wherein the graphical panels sent by the computing device to the displaydevice comprise: the first graphical panel comprising a resized inputvideo; the second graphical panel comprising local video signalsobtained from a local camera; a third graphical panel comprising dataobtained from a local security system; and a fourth graphical panelcomprising data obtained from an external data network.
 4. The method ofclaim 1, further comprising using the computing device to conduct ajoint content viewing session with a remote user, wherein the jointcontent viewing session comprises shared viewing of image content storedon a memory device connected to a first computing device of oneparticipant to the joint content viewing session.
 5. The method of claim1, further comprising: receiving, by the computing device, an inputvideo and one or more video overlay graphics; and combining, by thecomputing device, the one or more video overlay graphics with the inputvideo to generate the signal, wherein the signal comprises the inputvideo and the one or more video overlay graphics in separate graphicalpanels.
 6. The method of claim 1 further comprising: manipulating thefirst graphical panel associated with the first application in responseto the pre-defined command.
 7. The method of claim 1, furthercomprising: sending, by the computing device, an alternate displaysignal to a secondary computing device, wherein the alternate displaysignal comprises outlines of the first and second graphical panels andomits interior details of the first and second graphical panels; anddisplaying the alternate display signal on the secondary computingdevice and using rearrangement commands inputted to the secondarycomputing device to rearrange the first and second graphical panels onthe display device.
 8. The method of claim 1, wherein the computingdevice receives premises security system data from a premises securitysystem located at a premises of the computing device, and wherein thepre-defined command causes the first graphical panel to display thepremises security system data.
 9. At least one non-transitorycomputer-readable medium having computer-executable instructions storedthereon that, when executed, cause at least one processor to: send, by acomputing device and to a display device, a signal comprising at least afirst and a second graphical panel associated with at least a first andsecond application, respectively; define a three-dimensional space infront of a user and the display device, wherein the three-dimensionalspace is divided into a plurality of subspaces comprising at least afirst subspace and a second subspace, wherein the first subspace iscloser to the display device than the second subspace; associate thefirst and second applications to the first subspace and the secondsubspace, respectively; detect a gesture made by the user in the firstsubspace, wherein the gesture is associated with a pre-defined command;and send the pre-defined command associated with the gesture to thefirst application corresponding to the first subspace in which thegesture was detected.
 10. The at least one non-transitorycomputer-readable medium of claim 9, having additionalcomputer-executable instructions stored thereon that, when executed,further cause the at least one processor to: manipulate the firstgraphical panel associated with the first application in response to thepre-defined command.
 11. The at least one non-transitorycomputer-readable medium of claim 9, having additionalcomputer-executable instructions stored thereon that, when executed,further cause the at least one processor to: detect a second gesturemade by the user in the second subspace, wherein the second gesture isassociated with a second pre-defined command; send the secondpre-defined command associated with the second gesture to the secondapplication corresponding to the second subspace in which the secondgesture was detected; and manipulate the second graphical panelassociated with the second application in response to the secondpre-defined command.
 12. The at least one non-transitorycomputer-readable medium of claim 9, having additionalcomputer-executable instructions stored thereon that, when executed,further cause the at least one processor to: send, by the computingdevice, an alternate display signal to a secondary computing device,wherein the alternate display signal comprises outlines of the first andsecond graphical panels and omits interior details of the first andsecond graphical panels; and display the alternate display signal on thesecondary computing device and use rearrangement commands inputted tothe secondary computing device to rearrange the first and secondgraphical panels on the display device.
 13. A computing device,configured to: send, by the computing device and to a display device, asignal comprising at least a first and a second graphical panelassociated with at least a first and second application, respectively;define a three-dimensional space in front of a user and the displaydevice, wherein the three-dimensional space is divided into a pluralityof subspaces comprising at least a first subspace and a second subspace,wherein the first subspace is closer to the display device than thesecond subspace; associate the first and second applications to thefirst subspace and the second subspace, respectively; detect a gesturemade by the user in the first subspace, wherein the gesture isassociated with a pre-defined command; and send the pre-defined commandassociated with the gesture to the first application corresponding tothe first subspace in which the gesture was detected.
 14. The computingdevice of claim 13, further configured to: manipulate the firstgraphical panel associated with the first application in response to thepre-defined command.
 15. The computing device of claim 13, furtherconfigured to: detect a second gesture made by the user in the secondsubspace, wherein the second gesture is associated with a secondpre-defined command; send the second pre-defined command associated withthe second gesture to the second application corresponding to the secondsubspace in which the second gesture was detected; and manipulate thesecond graphical panel associated with the second application inresponse to the second pre-defined command.
 16. The computing device ofclaim 13, further configured to: send, by the computing device, analternate display signal to a secondary computing device, wherein thealternate display signal comprises outlines of the first and secondgraphical panels and omits interior details of the first and secondgraphical panels; and display the alternate display signal on thesecondary computing device and use rearrangement commands inputted tothe secondary computing device to rearrange the first and secondgraphical panels on the display device.